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REAL-TIME THREE-DIMENSIONAL IMAGE PROCESSING SYSTEM 
FOR NON-PARALLEL OPTICAL AXIS AND METHOD THEREOF 
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15 

TECHNICAL FIELD 

The present invention relates to an image processing system, and 
more particularly, to a real-time three-dimensional image processing system 
and a method with non-parallel optical axis cameras. 

20 

BACKGROUND ART 

Generally, a real-time three-dimensional image processing system 
employs a processor having a stereo matching as a main part. At this time, 
a process for recreating space information of three dimension space from a 
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pair of two-dimensional images is called as the stereo matching. 

In a research treatise (Uemsh R.Dhond and J.K.Aggarwal. Structure 
from Stereo-a review. IEEE Transactions on Systems, Man, and Cybernetics, 
19(6):553-572, nov/dec 1989), basic principle for the stereo matching is 
5 described in accordance with the conventional art employing the processor. 
Also, the substantiated art of the stereo matching is disclosed in a real-time 
three-dimensional image matching system (Korean Patent Application 2000- 
41424). 

The system according to the conventional art comprises a pair of 
1 0 cameras having the same optical characteristics. If the pair of cameras lighten 
a same space region, similar space regions are respectively selected to each 
horizontal image scan line of the cameras. Accordingly, in the same way that 
pairs of pixels of the scan lines correspond to each point of the thee- 
dimensional space, pixels in one image are matched to those in another 
15 image. By using a simple geometrical characteristic, a distance from the pair 
of cameras to a point in the three-dimensional space can be measured. 
Herein, a difference between a position of a predetermined pixel in an image 
selected from one camera and a position of a predetermined pixel 
corresponding to an image selected from the other camera is called as a 
20 disparity. Also, the geometrical characteristic calculated from the disparity is 
called as "depth". That is, the disparity comprises distance information. 
Accordingly, if the disparity value is calculated from inputted images real-time, 
three-dimensional distance information and form information of an observation 



2 



space can be measured. 

However, in the system according to the conventional art, the disparity 
value is calculated to recognize space information only in a state that two 
cameras are parallel put. When a near object is observed, the object is not 
5 observed in an optimum state by said method. That is, when a far object is 
observed, if angles formed at a pair of cameras are parallel, the disparity is 
not great, thereby having no problem. However, when a near object is 
observed in a state that the angles formed at the pair of cameras are parallel, 
a measured disparity is too great or exceeds a measurement range of the 
10 system and an observation object is not normally reflected on each image of 
the parallel cameras, thereby having a problem in the image matching. 

Actually, when the system is realized with an application specific 
integrated circuit-chip (ASIC-chip), the real-time three-dimensional image 
matching system in accordance with the conventional art had a problem that 
15 a space of a memory unit occupies many parts in an entire processor. 

Also, a forward processor and a backward processor are alternately 
operated in the system. Accordingly, when one processor is operated, the 
other processor is obliged to stand idle, thereby not being efficient and having 
a slow processing speed. 

20 

DISCLOSURE OF THE INVENTION 

Therefore, an object of the present invention is to provide a system for 
calculating a position and a form in three-dimensional space and a method 
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thereof, in which an observation is facilitated by controlling a camera angle 
according to a position of an object, and a disparity value is prevented from 
being overflowed above a predetermined value. 

Also, another object of the present invention is to provide a system for 
5 controlling a reference offset value of an outputted disparity and a method 
thereof, in which if it is assumed that the uppermost processing element 
represents the maximum disparity value, the lowermost processing element 
represents the minimum disparity value, and a base processing element has 
'0' as a disparity value, a position of the base processing element is properly 
10 set. 

Also, still another object of the present invention is to provide a system 
which reduces a fabricating cost by replacing the conventional memory unit 
by a cheap external memory device and a method thereof. 

Also, still the other object of the present invention is to provide a 
15 system which can obtain a performance faster than the conventional art more 
than two times by alternately storing a processed decision value in one 
memory device between two memory devices and thereby consecutively 
operating forward and backward processors and a method thereof. 

The foregoing and other objects, features, aspects and advantages 
20 of the present invention will become more apparent from the following 

detailed description of the present invention when taken in conjunction with 
the accompanying drawings. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



Figure 1 is a block diagram showing a real-time three-dimensional 
image processing system with non-parallel optical axis according to the 
present invention; 

5 Figure 2 is a detail view of an image matching unit of Figure 1 ; 

Figure 3 is a detail view of a processing element of Figure 2; 
Figure 4 is a detail view of a forward processor of Figure 3; 
Figure 5 is a detail view of a path comparator of Figure 4; 
Figure 6 is a detail view of a accumulated cost register of Figure 4; and 
10 Figure 7 is a detail view of a backward processor of Figure 3. 



MODE FOR CARRYING OUT THE PREFERRED EMBODIMENTS 

If it is assumed that a camera performs the same performance with a 
man's eyes, a pair of cameras can capture an image regardless of a distance 
15 in an optimum state by controlling a focus direction according to far and near. 
Accordingly, so as to change an observation eye of a camera according to the 
far and near, a means for controlling an angle of the camera and a means for 
renewing a setting of an image matching system according to a control of the 
angle are required. By said means, even a near object is well measured and 
20 more effective image matching is possible. 

The present invention will now be described with reference to 
accompanying drawings. 

Figure 1 is a block diagram showing a real-time three-dimensional 
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image processing system with non-parallel optical axis according to the 
present invention. The system in Figure 1 comprises a left camera 10 and a 
right camera 11 having optical axis rotations, an image processing unit 12 for 
temporarily storing digital image signals of the left and right cameras 10 and 
5 1 1 or converting an analogue image signal into a digital, thereby respectively 
outputting the digital image signals, an image matching unit 13 for calculating 
a decision value representing the minimum matching cost from the left and 
right digital image signals and then for outputting a disparity value according 
to the decision value, a user system 16 for displaying images by the disparity 
10 value, and first and second memory devices 14 and 1 5 for alternately storing 
the decision value so as to provide the decision value to the image matching 
unit 13. 

Herein, though a rotation axis of the cameras 10 and 11 is not 
illustrated in Figure 1, a cylindrical body (not shown) constituting a lens part 
15 (not shown) of the cameras 10 and 1 1 can be rotated or an entire camera 
body can be rotated as shown in Figure 1, of which detailed explanations will 
be omitted. 

The image processing unit 12 processes images of an object obtained 
from the left camera 10 and the right camera 11, and outputs digitally 
20 converted left and right images to the image matching unit 13 in the form of 
pixels. Then, the image matching unit 13 sequentially receives pixel data of 
each scan line of the left and right images, calculates a decision value of the 
left and right images, stores the calculated decision value in one memory 
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device between the first and second memory devices 14 and 15, and reads 
a previously stored decision value in the other memory device, in which the 
storage and reading are alternately performed. Therefore, a disparity value is 
calculated from the read decision value and outputted to the user system 16. 
5 Also, a process for outputting the disparity value is repeatedly performed for 
all pairs of scan line from the two images. 

Figure 2 is a detail view of the image matching unit of Figure 1. The 
image matching unit 13 in Figure 2 comprises N/2 left image registers 20 and 
N/2 right image registers 21 for respectively storing left and right image 

10 signals of the image processing unit 12, N processing elements 22 for 
calculating a decision value from images inputted from the left and right image 
registers 20 and 21 synchronous to the clock signals (CLKE, CLKO) and for 
outputting a disparity value (Dout), a decision value buffer 24 for alternately 
exchanging the decision value with the first and second memory devices by 

15 a selection signal, and a control unit 23 for controlling the processing 
elements 22 by setting signals (a top signal, a bottom signal, a base signal, 
and a reset signal) which set register values 43 of the processing elements 
22 by receiving an external control signal. 

A method for processing a pair of scan lines by the image matching 

20 unit will be explained. 

First, the control unit 23 receives the external control signal and 
outputs the top, bottom, base, and reset signals to the N processing elements 
22. At this time, the top signal is activated in the uppermost processing 
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element among the processing elements in a range of a disparity value, and 
the bottom signal is activated in the lowermost processing element. Also, the 
base signal is activated in a processing element of a proper position of which 
disparity value is '0' so as to optimize the disparity value between the 
5 processing element activated by the top signal and the processing element 
activated by the bottom signal according to an optical axis angle of the pair of 
cameras 10 and 1 1 by a distance from a subject. 

Herein, as shown in Figure 2, if it is assumed that a processing 
element (N-1) located at the uppermost position among the several 

10 processing elements 22 is defined as the uppermost processing element, a 
processing element (0) located at the lowermost position is defined as the 
lowermost processing element, and a disparity value in a position of the 
processing element in which the base signal is active is defined as '0', a 
disparity value below the disparity value of '0' becomes -1 and a disparity 

15 value below the disparity value of ' -1' becomes -2. That is, the uppermost 
processing element and the lowermost processing element have the minimum 
and the maximum of the disparity value. 

The image registers 20 and 21 receive pixel data of each scan line of 
left and right images digitally converted from the image processing unit 12 and 

20 output the pixel data to the processing elements 22. At this time, the 
processing elements 22 can be reproduced as a linear array form up to a 
preset maximum disparity value, and each processing element 22 can 
exchange information with adjacent processing elements. By said structure, 



the system can be operated with the maximum speed regardless of the 
number of the processing elements 22. 

The image registers 20 and 21 store image data of each pixel in each 
corresponding system clock, and each activated processing element calculate 
5 a decision value from the left and right images. At this time, the decision value 
buffer 24 alternately stores the calculated decision value calculated from the 
processing elements 22 in the first memory device 14 or the second memory 
device 15, and alternately reads the decision value from the first and second 
memory devices 14 and 15, thereby inputting to the processing elements 22. 

10 That is, the decision value buffer 24 stores the decision value calculated from 
the processing elements 22 in one memory device between the first and 
second memory devices 14 and 15, and inputs the decision value read from 
the other memory device to the processing elements 22 by a selection signal. 
Herein, the selection signal represents whether a data of the first memory 

15 device 14 is accessed or a data of the second memory device 15 is accessed. 

The processing elements 22 input the decision value alternately read 
from the first memory device 14 or the second memory device 15 by the 
decision value buffer 24, and compute a disparity value, thereby outputting to 
the user system 16. At this time, the disparity value can be outputted as a 

20 sensitization form such as the actual value, or as an offset relative to the 
previous disparity value. 

Herein, the image registers 20 and 21 and the processing elements 22 
are controlled by two clock signals (CLKE) (CLKO) derived from a system 



clock. The clock (CLKE) is toggled on an even- numbered system clock 
cycles (an initial system clock cycle is supposed as '0') and supplied to the 
image register 20 that store the right image and to even-numbered processing 
elements 22. Also, the clock signal (CLKO) is toggled on an odd-numbered 
5 system clock cycles and supplied to the image register 21 that store the left 
image and to odd- numbered processing elements 22. Accordingly, the image 
registers 20 or 21 and the even numbered or the odd numbered processing 
elements 22 are operated at each system clock cycle by starting from the 
image register 20 and the even numbered processing elements 22. 

10 Figure 3 is a detail view of the processing elements 22 of Figure 2. The 

processing element 22 in Figure 3 comprises a forward processor 30 for 
receiving scan line pixels stored in the image registers 20 and 21 and 
outputting an accumulated matching cost to the adjacent processing elements 
and a decision value to the decision value buffer 24, and a backward 

15 processor 31 for receiving the decision value (Dbin) outputted from the 
decision value buffer 24 and outputting a disparity value. 

The operation of the processing element 22 will be explained in detail. 
The processing element 22 is initialized by a reset signal in which an 
accumulated cost register value of the forward processor 30 and an active 

2 0 register value of the backward processor 31 are initiated. That is, if an active 
base signal is inputted to the processing element at first, the accumulated 
cost register value of the forward processor 30 becomes '0' and the active 
register value of the backward processor 31 becomes '1'. On the contrary, if 



a base signal which is not active is inputted to the processing element at first, 
the accumulated cost register value of the forward processor 30 is initialized 
to nearly maximum value can be represented by the active register and the 
active register value of the backward processor 31 is initialized to '0\ 
5 The forward processor 30 calculates a decision value (Dcout) by 

processing a left and right images synchronous to one of the clock signals 
(CLKE) (CLKO), and stores the decision value (Dcout) in the first memory 
device 14 or in the second memory device 15 through the decision value 
buffer 24. 

10 The backward processor 31 operates the decision value read from the 

first memory device 14 or the second memory device 15 through the decision 
value buffer 24 and calculates a disparity value, thereby outputting the 
disparity value synchronous to one of the clock signals (CLKE) (CLKO). At 
this time, while the decision value calculated from the forward processor 30 

15 is written in the one memory device between the first and second memory 
devices 14 and 15, the decision value is inputted to the backward processor 
31 in the other memory device. 

Then, when a next scan line is processed, the forward processor 30 
converts the memory devices 14 and 15 for storing the decision value (Dcout) 

20 by inverting the selection signal, and the backward processor 31 also reads 
the decision value from the inverted memory devices 14 and 15, thereby 
repeating the above processes. 

Figure 4 is a detail view of the forward processor 30 of Figure 3. The 
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forward processor 30 in Figure 4 comprises an absolute difference value 
calculator 40 for calculating an image matching cost through the absolute 
value of the difference of two pixels of the scan lines outputted from the image 
registers 20 and 21 , a first adder 41 for adding the matching cost calculated 
5 from the absolute difference value calculator 40 to an accumulated cost fed- 
back from an accumulated cost register 43 which will be later explained, a 
path comparator 42 for receiving the output value from the first adder 41 and 
the accumulated costs of the adjacent processing elements 22, and the top 
and bottom signals and outputting the constrained minimum accumulated 

10 cost, an accumulated cost register 43 for storing the minimum accumulated 
cost outputted from the path comparator 42 as the accumulated cost, and a 
second adder 44 for adding the accumulated cost stored in the accumulated 
cost register 43 to an occlusion cost and outputting the summed cost to the 
adjacent processing elements 22. 

15 Herein, the base signal and the reset signal initialize the accumulated 

cost register 43. 

Figure 5 is a detail view of the path comparator 42 of Figure 4. The 
path comparator 42 in Figure 5 comprises the occlusion comparator 50 and 
the comparator 51. 

20 The occlusion comparator 50 comprises a comparator 52 for 

comparing an up occlusion path accumulated cost (uCost) with a down 
occlusion path accumulated cost (dCost) and outputting the minimum cost 
input (up and down), a multiplexer (MUX) 53 for selecting the up occlusion 
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path accumulated cost or the down occlusion path accumulated cost and 
outputting to the comparator 51, an AND gate 54 for performing an AND 
operation by receiving the bottom signal and an output of the comparator 52, 
and an OR gate 55 for operating the multiplexer 53 by performing an OR 
5 operation for the top signal and an output of the AND gate 54. 

The comparator 51 selects the minimum cost input between an 
outputted minimum occlusion cost from the occlusion comparator 50 and 
output (mCost) of the first adder 41, thereby outputting the minimum 
accumulated cost (MinCost) and the "match path decision". 

10 The path comparator 42 prevents the up occlusion path accumulated 

cost (uCost) from being selected when the top signal notifying the up 
processing elements activated, prevents the down occlusion path 
accumulated cost (dCost) from being selected when the bottom signal is 
activated, and in other cases, selects the minimum cost among the up 

15 occlusion path accumulated cost (uCost), down occlusion path accumulated 
cost (dCost), and the added cost (mCost). That is, the comparator 52 outputs 
two values by comparing two inputs (uCost, dCost). At this time, the upper 
output (MinCost) represents the minimum value and the lower output 
indicates which is the minimum among the inputted values. 

2 0 The multiplexer 53 selects one value between the two inputted values 

(uCost, dCost) by an output value of the OR gate 55, thereby outputting. 
Operations of the forward processor 30 will be explained in detail. 
First, when the top signal is active, the path comparator 42 excludes 
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the up occlusion path accumulated cost among the up occlusion path 
accumulated cost, the down occlusion path accumulated cost, and the added 
cost, and compares only the down occlusion path accumulated cost with the 
added cost, thereby outputting the minimum cost. At this time, if the down 
5 occlusion path accumulated cost is the minimum value, a decision value of '-T 
is outputted, and if the added cost (mCost) is the minimum value, a decision 
value of '0' is outputted. Herein, if the decision value is 2bits, '11' corresponds 
to -1 , '00' corresponds to 0, and '01 ' corresponds to +1 . When the top signal 
is active, the OR gate 55 to which the top signal is inputted outputs an up bit 

10 (Dcout (1) = Dfout (1)) of the decision value as T and the multiplexer 53 
selects the down occlusion path accumulated cost by the decision value 
(Dccout) to output to the comparator 51. Therefore, the comparator 51 
compares the down occlusion path accumulated cost with the added cost and 
outputs the minimum cost. 

15 Also, in case that the bottom signal is active, the path comparator 42 

excludes the down occlusion path accumulated cost among the up occlusion 
path accumulated cost, the down occlusion path accumulated cost, and the 
added cost, and compares only the up occlusion path accumulated cost with 
the added cost, thereby outputting the minimum cost and a decision value 

20 (Dbin). Since the active bottom signal is inverted and inputted to an input 
terminal of the other side of the AND gate 54, an output signal of the AND 
gate 54 becomes '0'. Also, since the top signal is , 0', an up bit (Dcout (1) = 
Dfout (1)) of the decision value outputted from the OR gate 55 is outputted as 



Accordingly, since the multiplexer 53 selects the up occlusion path 
accumulated cost and inputs to the comparator 51, the comparator 51 
compares the up occlusion path accumulated cost with the added cost to 
output the minimum cost. 

Also, in case that neither the top signal nor the bottom signal is active, 
the path comparator 42 outputs the minimum cost among the up occlusion 
path accumulated cost, the down occlusion path accumulated cost, and the 
added cost, and outputs the decision value (Dcout). 

The minimum cost outputted by the path comparator 42 becomes a 
new accumulated cost synchronous to the clock signal (CLKE or CLKO) by 
storing it in the accumulated cost register 43. 

Figure 6 is a detail view of the accumulated cost register 43 of Figure 
4. The accumulated cost register 43 in Figure 6 receives an input of the path 
comparator 42, and comprises edge-triggered D-flip flops 62 and 63 which are 
set or cleared synchronous to the clock signal (CLKE or CLKO) when a reset 
signal is activated, and a demultiplexer 61 for selecting whether the D-flip flop 
will be set or cleared according to the base signal. 

Herein, the D-flip flop 63 is not set by a fixed value '1' but reset only by 
the reset signal. 

Operations of the accumulated cost register 43 will be explained. 
At a down position of the D-flip flop 62, predetermined numbers of bits 
among the minimum cost (MinCost = U[i,j])are stored, and at an up position 
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of the D-flip flop 63, predetermined numbers of bits are stored. The 
demultiplexer 61 inputs the set signal or the reset signal to the D-flip flop 62 
by the base signal by receiving the reset signal. 

The D-flip flop 63 is not set by a fixed value '1' but reset only by the 
5 reset signal. An output signal (U[i-1 J]) of the D-flip flops 62 and 63 is 
outputted to the second adder 44. The second adder 44 adds the occlusion 
cost (y) to the accumulated cost stored in the accumulated cost register 43, 
and outputs the summed value (Uout) to adjacent processing elements. The 
occlusion cost (y) is a constant value. 

10 Figure 7 is a detail view of the backward processor 31 of Figure 3. The 

backward processor 31 in Figure 7 comprises a demultiplexer 73 that directs 
the reset signal to the set or clear input of the active register according to 
base, an active register 71 composed of D-flip flops which are set or cleared 
by the output of the demultiplexer 73, an OR gate 70 for performing a logical 

15 OR operation using the active bit paths (Ain1 , Ain2 and Aself) as inputs and 
outputting the result to the active register 71 , a demultiplexer 72 for outputting 
an output value of the active register 71 according to the decision value 
(Dbin), and a tri-state buffer 74 for outputting the decision value (Dbin) under 
the control of the output of the active register 71 . 

20 Operations of the backward processor 31 will be explained. 

The tri-state buffer 74, when an input value is T, outputs the input 
value as it is, and in other cases, does not output anything as the tri-state 
buffer becomes a high impedance state. 
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When the active register 71 has a value of 1 1\ the tri-state buffer 74 
outputs the input value (Dbin), and when the active register 71 has a value of 
'0\ the output of the tri-state buffer is placed in the high impedance state. 
The OR gate 70 performs a logical OR operation using three inputs; 
5 the active bit paths (Ain1 , Ain2) of the adjacent processing elements 22 and 
the fed-back active bit path (Aself). The result is outputted to the active 
register 71. The input terminal (Ain1) is connected to an output terminal 
(Aout2) of a downwardly adjacent processing element, and the input terminal 
(Ain2) is connected to an output terminal (Aout2) of an upwardly adjacent 
10 processing element. The input terminals Ain1 and Ain2 represent paths by 
which an active bit datum output from the active register 71 of adjacent 
processing elements can be transmitted. Accordingly, if the active bit (Aself) 
is a high state, an output signal of the OR gate 70 becomes a high state. 
The input signals (Ain1, Ain2) maintain a state of the active bit in the 
15 active register 71 when the clock is applied to the path of the active bit, and 
a new value of the active bit is stored into the active register 71 when the 
clock is applied to the backward processor 31. 

The demultiplexer 72 is controlled by the decision value (Dbin) read 
from the first and second memory devices 14 and 15. The output signals 
2 0 (Aoutl , Aself and Aout2) of the demultiplexer 72 have the same value as the 
ouput of the active bit when the decision values (Dbin) are -1, 0, and +1, 
respectively, otherwise they are '0\ 

The tri-state buffer 74 outputs the decision value (Dbin) as a disparity 
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value (Dbout = Dout) when the output of the active register 71 is '1'. If the 
output of the active register 71 is '0', the output (Dbout) of the tri-state buffer 
74 is placed in a high impedance state, thereby avoiding any conflict with the 
output (Dbout) of the processing element. 

Also, the disparity value can be outputted instead of the decision value 
(Dbin), which represents an actual disparity value differently from a case that 
the disparity value is relatively changed by outputting the decision value 
(Dbin). 

In the meantime, the algorithm for matching each pixel in pairs of the 
scan lines according to preferred embodiments of the present invention will 
be explained. 

The control unit 23 sets the top signal, the bottom signal, and the base 
signal as follows. 

A number of a processing element in which the top signal is activated: 

Jtop 

A number of a processing element in which the bottom signal is 
activated: j B07T0M 

A number of a processing element in which the base signal is 
activated:/^ 

0 - Jtop ~ J base - J bottom ~ N-1 

Herein, U[i,j] is the accumulated cost register 43 value of the forward 
processor 30 of the j th processing element in i m clock cycle. That is, the U[i,j] 
is an accumulated cost register 43 value of the j th forward processor 30 in i th 
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step. 

First, the initialization operation will be explained. 

In initializing the system of the present invention, the accumulated 
costs of all the accumulated cost registers except j B ASE lh accumulated cost 
5 register are set to a value (oo) that is nearly the maximum value that can be 
represented. 

That is, \J[0,j BASE ] = 0, 

U[0, j] = oo, herein, je{0, j BASE . ]t j BASE+x , .... N-1} 

Then, operations of the forward processor and the backward processor 
10 will be explained. 

The forward processor searches the best path and cost by using the 
following algorithm for each step i and each processing element j. 
For i = 1 to 2N do; 

For each j e {0 N-1}: 

15 ifi+jiseven: 

U[i,j]= min i€{ . W)J+keljBoT jnp) U[i-l,j + k] + rk 2 

P M [i,j]=arg min kmt - W) j+t. Um .M U[i + k] + rk 2 
if i+j is odd: 

U[ij] = U[i - 1, j] +\g'[(i-j + \)/2)-g r [(/ + j + l)/2] | 

20 P u [i,fl s 0 

Herein, P M and Pm respectively correspond to the first memory device 
14 and the second memory device 15, or to the second memory device 15 

19 



and the first memory device 14, and stores the decision value which is an 
output value of the forward processor 30. g' [i], g r [i] represents i th pixel value 
on the same horizontal lines of the left and right images, respectively. Also, 
y is the occlusion cost in a case that predetermined pixels in one image do not 
5 correspond to predetermined pixels to be matched in another image. The y is 
defined by a parameter. 

For example, the forward processing method in the third processing 
element in the fifth clock will be explained. 

In the fifth clock and in the third processing element, the sum of 5 and 

10 3 is an even number, so that the accumulated cost register value of the up 
processing element (the accumulated cost register value of the fourth 
processing element), the accumulated cost register value of the down 
processing element (the accumulated cost register value of the second 
processing element), and its own accumulated cost register value (the 

15 accumulated cost register value of the third processing element) are 
respectively compared to obtain a processing element having the minimum 
cost. If the accumulated cost register value of the up processing element is 
determined as the minimum cost, '+1' is outputted as the decision value, and 
if the accumulated cost register value of the down processing element is 

20 determined as the minimum cost, -1' is outputted as the decision value. 
Finally, the accumulated cost register value of the its own accumulated cost 
register value is determined as the minimum cost, '0' is outputted as the 
decision value. 
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Also, if a sum between the number of times of the clock and a number 
of the processing element is an odd number, the decision value is '0'. 
However, in that case, information for the i th pixel value on the same 
horizontal line in the left and right images is included, thereby including image 
5 information which was not represented at the forward processor step. 

The backward processor generates the disparity value and outputs by 
the decision value which is a result of the forward processor through the 
following algorithm. 

For i = 1 to 2N do; 

10 D[i-1] = d[i]+P M [i,d(l)] 

Herein, PM'[i.d(i)] represents a decision value outputted through the 
backward processor having the activated bit of '1' in the i th clock by reading 
from the first memory device or the second memory device. 

The active register 71 is initialized at first by the reset signal and the 
15 base signal which are activated by the control unit 23. The decision value 
outputted from the forward processor 30 is stored in the Pm [i,j], at the same 
time, the backward processor 31 reads the decision value (Dout) of P M '[i.j] 
stored in the previous scan lines, and the P M [i,j] and the P M '[i j] correspond 
to the first and second memory devices 14 and 15 as stacks having a 
20 structure of last in first out (LIFO). 

Also, when the forward processor and the backward processor which 
are performed at the same time are finished, the P M [i,j] and the P M '[iJ] are 
respectively changed into the second memory device 1 5 and the first memory 
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device 14 to process a next processing. If the processing is finished, a role is 
again changed. 

The forward processor and the backward processor are in parallel 
processed by using a processing element. 

5 

INDUSTRIAL APPLICABILITY 

As so far described, in the present invention, a position and a form in 
three-dimensional space can be calculated by facilitating an observation by 
controlling a camera angle according to a position of an object, and a disparity 

10 value is prevented from being overflowed above a predetermined value. 

Also, whereas the disparity value had a constant range of amount in 
the conventional system, in the present invention the disparity value had 
different ranges fit to a measurement range according to an angle of a camera 
optical axis. That is, if it is assumed that the uppermost processing element 

15 represents the maximum disparity value, the lowermost processing element 
represents the minimum disparity value, and a base processing element has 
'0' as a disparity value, a position of the base processing element is properly 
set, thereby controlling a base offset value of the outputted disparity, that is, 
a size value. 

20 Also, in the present invention, the maximum and the minimum ranges 

of the disparity are limited by a setting of the uppermost, the lowermost, and 
the base processing element Accordingly, the disparity value range limiting 
means is further included so as to prevent a wrong disparity output when the 
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disparity range is exceeded by noise generated at an external environment. 
Actually, when the system is realized with ASIC chip, in the real-time three- 
dimensional image matching system according to the conventional art, a 
space of a memory unit occupies many parts in the entire processor. 
5 However, in the present invention, a fabricating cost is reduced by replacing 
the conventional memory unit by a cheap external memory device. 

Also, in the present invention, two external memory devices having the 
stack performances are added. Accordingly, while the forward processor 
stores the processed decision value into the first external memory device, the 

10 backward processor reads the stored decision value from the second external 
memory device, and when next image scan lines are processed, while the 
forward processor stores the processed decision value into the second 
external memory device, the backward processor reads the stored decision 
value from the first external memory device. Therefore, the system alternately 

15 stores the processed decision value into the one memory device between the 
two memory devices, so that the forward and backward processors are 
consecutively operated, thereby having a faster performance more than two 
times than the conventional art. 
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